1. Technical Field
This invention relates to storage systems. More specifically, this invention relates to management of disk drive modules in the redundant array of independent disk system.
2. Description of the Prior Art
Redundant Arrays of Independent Disks (RAID) is a taxonomy of redundant disk array storage schemes defining a number of ways of configuring and using multiple computer disk drive modules (hereinafter “drives”) to achieve varying levels of availability, performance, capacity, and cost, while appearing to the software application as a single large capacity drive. Conventional RAID storage systems can be implemented in either by means of hardware or software. In the former instance, RAID algorithms are packaged into separate controller hardware coupled to the computer input/output (I/O) bus. The software implementation incorporates RAID algorithms into system software executed by a processor together with the operating system, and obviates the need and cost of a separate hardware controller.
RAID systems employ controllers to link a host computer with one or more storage devices in the RAID. These controllers handle management of the disks in the RAID. FIG. 1 is an exemplary computer system in which a RAID, or other disk array, is implemented. An apparatus (10) may represent practically any type of computer, computer system, or other programmable electronic device, including a client computer, a server computer, a portable computer, a handheld computer, an embedded controller, a personal digital assistant, etc. In one embodiment, an apparatus (10) may be implemented using one or more networked computers, e.g. in a cluster or other distributed computing system. The apparatus (10) will hereinafter also be referred to as a “computer” although it should be appreciated that the term “computer” may also include other suitable programmable electronic devices consistent with the invention.
A computer (10) typically includes at least one processor (12) coupled to memory (14). A processor (12) may represent one or more processors, and memory (14) may represent random access memory devices comprising the main storage of the computer (10), as well as any supplemental levels of memory, e.g. cache memory, non-volatile memory, backup memory, read-only memory, etc. In addition, memory (14) may be considered to include memory storage physically located elsewhere in the computer (10), e.g. any cache memory in a processor (12), as well as any storage capacity used as a virtual memory, e.g., as stored on the disk array (34) or on another computer coupled to the computer (10) via a network (18).
The computer (10) typically receives a number of inputs and outputs for communicating information externally. For interface with a user or operator, the computer (10) typically includes one or more user input devices (22), e.g. a keyboard, a mouse, a trackball, a joystick, a touchpad, a microphone, etc., and a display (24), e.g. a monitor, a display panel, a speaker, etc. Otherwise, user input may be received via another computer interfaced with the computer (10) over the network (18) via a dedicated workstation interface or the like. For additional storage, the computer (10) may also include one or more mass storage devices accessed via a storage controller, or adapter, (16). The mass storage devices may take the form of a removable disk drive, a hard disk drive, a direct access storage device, an optical drive, a tape drive, etc. In addition, the computer (10) may include an interface with one or more networks (18), e.g. a LAN, a WAN, a wireless network, etc., to permit the communication of information with other computers coupled to the network. It should be appreciated that the computer (10) typically includes suitable analog and/or digital interfaces between a processor (12) and each of the components (14), (16), (18), (22), and (24) as is well known in the art.
A computer (10) operates under the control of an operating system (30), and executes software programs (32) or otherwise relies upon various computer software applications, components, programs, objects, modules, data structure, etc. Moreover, various applications, components, programs, objects, modules, etc. may also execute on one or more processors in another computer coupled to the computer (10) via a network (18), e.g., in a distributed or client-server computing environment, whereby the processing required to implement the functions of a computer program may be allocated to multiple computers over a network.
FIG. 2 is a block diagram (100) of a prior art disk array system illustrating configuration of array sites in a RAID. In this illustration, two storage enclosures are shown (102) and (104), although in one embodiment, there may be more storage enclosures. Each storage enclosure (102) and (104) houses a plurality of disk drive modules, from a larger storage enclosure (108), with each disk drive module therein generally of the same storage capacity and RPM. In one embodiment, an enclosure may house sixteen disk drive modules. The disk drive modules in each enclosure (102) and (104) are in communication with the storage controller (116), hereinafter referred to as a device adapter, through an electrical connection known as a dual loop. As shown, storage enclosure (102) is connected to the device adapter (116) through a first dual loop (112), and storage enclosure (104) is connected to the device adapter (116) through a second dual loop (114). All disk drive modules in the storage enclosure (102) belong to the first dual loop (112), and all disk drive modules in the storage enclosure (104) belong to the second dual loop (114). An array site (120) is a grouping of eight disk drive modules, with four disk drive modules selected from the first dual loop (112) and four disk drive modules selected from the second dual loop (114), i.e. the array site will have four disk drive modules each from storage enclosure (102) and storage enclosure (104). A RAID is created with a minimum of one array site (120), wherein each array sites forms an array (122). Array (122) is a logical form of array site which is used for managing and maintaining the RAID efficiently. At the time of creation of the RAID (124), some of the disk drive modules in each array site may be designated as spare modules. Furthermore, each array site will be assigned to a RAID. Array sites are assigned to a RAID during creation of the RAID. If the array site is not assigned to a RAID, the array site is classified as unassigned and may be maintained for creation of future RAIDs.
It has become accepted in the art to have a minimum number of spare disk drive modules on each array site designated as spare modules, with spare modules originating from the first dual loop and the second dual loop in communication with the device adapter, as disk drive modules tend to fail for various reasons. When a drive fails, it needs to be either repaired or replaced, and the RAID needs to continue operating without loss of data or any apparent affect on the operation of the RAID. Prior art processes are provided to check for availability of spare disk drive modules on a single unassigned array site. If more than one spare disk drive module exists on each dual loop on the unassigned array site, the spare modules are removed from the array site. However, there are drawbacks with the prior art process for freeing up spare modules in an array site. Such drawbacks include complexity and inefficiency of the algorithm, as well as the inability to fulfill the standard of maintaining a minimum number of spare modules for a pair of dual loop connections of an assigned array site to a single device adapter.
Accordingly, there is a need to efficiently control the quantity and availability of spare disk drive modules at an array site. A process should address the limitations of the prior art and enable the RAID to continue operating with confidence that one appropriately configured modules be available.
FIG. 3 is a flow chart (200) illustrating a prior art process for determining if any drives in a RAID have failed and attending to repair and/or replacement of the failed drive. When a RAID member drive fails, the controller automatically swaps the failed drive with a spare drive and initiates data recovery. A routine is invoked at preset time intervals to evaluate the status of the RAID. In one embodiment, the routine may be in the form of a script or shell script, and the routine is programmed to execute once every hour. There are three primary elements to the routine, determining if any disk drives have been rejected or failed by the RAID adapter, determining if any disk drives have been bypassed, and determining if a failed disk drive is in need of immediate repair. In the context used herein, a bypassed drive is a drive that is electrically isolated from rest of the system. For example, a drive gets bypassed if the performance is degraded or if the drive is pulled out of the system. Once the routine is started (202), the initial test is to determine if any disk drives in the RAID have been rejected or failed (204). A negative response to the test at step (204) will result in a subsequent test to determine if any disk drives in the RAID have been bypassed (208). A positive response to either of the tests at steps (204) and (208) results in a subsequent test to determine if the failed or bypassed drive is in need of immediate repair (206). If the response to the test at step (206) is positive, the error(s) for the disk drive(s) identified at step (206) are entered into an error log and a disk drive management routine is called to update the object model (210), i.e. to designate the failed drive as failed spare and the spare drive as RAID member. Thereafter, a service event is received by a hardware management console (212), and the drive is unconfigured (214). In one embodiment, a service event may be an error log. In the context used herein, unconfigure is defined as removing the drive entry from Object Data Manager (ODM), checking the effect of repairing that drive on the system, and isolating the drive for repair. Thereafter, the failed module is replaced with a new module, and the new module is configured (216) and formatted (218) to make it available within the array. Following step (218) or a negative response to the tests at step (206) or (208), the routine returns to step (220) to wait for a set period of time before returning to step (202). In one embodiment, the set period of time is sixty minutes. Accordingly, the above-outlined routine identifies failed drives and attends to repair and/or replacement thereof.
However, there are drawbacks with the prior art repair and/or replacement routine. One such drawback is the scheduling of the routine. The prior art routine is not automated and only operates at preset times. If one of the disk drives fails in between a sequential set of routines, the failure of the disk drive will not be detected and reported until the running of the next routine. In addition, the repair process of a failed disk drive is not prompt. A person must physically review the error log generated by the routine and attempt to repair the failed drive causing a delay in the repair process. In addition, the prior art does not address failure of multiple disk drives during the execution of this routine and identification of such during a single routine execution. This could give rise to a situation where spare disk drive gets exhausted and multiple RAIDS may be degraded. If there is an insufficient quantity of spare disk drives available at any time, then the RAID system can become inefficient or cause loss of data. Finally, the process of running a device health check on an hourly basis, or a different preset time interval utilizes processor and memory resources. This affects the physical performance of the associated computer and the array of disk drives.
Accordingly, there is a need to automate the process of identifying and repairing faulty disk drives in a RAID. An automated process should address the limitations of the prior art and enable the RAID to continue operating in an uninterrupted manner during a repair and/or replacement process.
It is also known in the art that spare disk drive modules are used to replace a failed or failing disk drive module. As such, it is critical that a spare module be available in a RAID with the capacity and RPM matching or greater than the failed module. One know prior art algorithm for creating spare disk drive modules assigns numerical identifiers to each array site and then divides the array sites based upon whether the numerical identifier is an odd number or an even number. Spare disk drive modules are created on array sites with an odd identifier from the first dual loop, and spare disk drive modules are created on array sites with an even identifier from the second dual loop.
However, there are drawbacks associated with this prior art algorithm. For example, array sites are created in a simultaneous manner and are continuously manipulated. There is likelihood that the assignment of an even distribution of odd and even numbered array sites to a device adapter may not be equitable. This would result in creation of spare disk drive modules on such array sites in a manner that reflects the arbitrary nature of assignment of odd and even numerical identifiers to the array site and creation of spare disk drive modules based on this arbitrary assignment.
Accordingly, there is a need to create an algorithm that assigns spare disk drive modules to unassigned array sites based upon the operating characteristics of both the module and the array site. The algorithm should address the limitations of the prior art and enable the assignment to ensure that a failed disk drive module can be temporarily or permanently replaced by a spare module with a capacity and speed that can match or exceed that of the failed or failing disk drive module.
In response to failure and replacement of disk drive modules, an array site and a RAID may become imbalanced due to unavailability of a perfect matching spare drive for replacement. There are drawbacks with the lack of uniformity among the disk drive modules in the RAID. Such drawbacks may include reducing the performance of the RAID either by capacity, interface rate, and/or RPM, as well as waste of resources.
Accordingly, there is a need to efficiently control the assignment of disk drive modules to an array site during operation thereof. The process should address the desire to establish a uniform array site while allowing the RAID to continue operating with confidence of the availability of appropriately configured modules.